Nested-Batch-Mode Learning and Stochastic Optimization with An Application to Sequential MultiStage Testing in Materials Science

We consider the nested-batch decision problem where we need to make a first stage choice (e.g. the size of a nanoparticle) after which we then need to run a series of experiments in batch selecting several second stage choices (e.g. testing different densities of the nanoparticle). Since these experiments are time consuming and expensive, we propose to estimate the value of information from the choice of the first stage decision (the size), to help guide the scientist in the selection of the next batch of experiments to run. The batch experiments are designed assuming that we maximize the value of information for an entire batch. The value of information, known as the Knowledge Gradient, requires calculating the expected maximum of a function. Since the calculation of the expected maximum is computationally intractable, we propose a Monte Carlo-based approach to address this hurdle in the context of both the batch and nested-batch problems. We empirically demonstrate the effectiveness of our approach on the material design problem of maximizing output current of a photoactive device, where it is competitive with a fully sequential optimal learning strategy and significantly outperforms pure exploration, pure exploitation and ε-greedy strategies with regard to the opportunity cost metric (8.1).